Arbitrary high-order linearly implicit energy-preserving algorithms for Hamiltonian PDEs

In this paper, we present a novel strategy to systematically construct linearly implicit energy-preserving schemes with arbitrary order of accuracy for Hamiltonian PDEs. Such novel strategy is based on the newly developed exponential scalar variable (ESAV) approach that can remove the bounded-from-blew restriction of nonlinear terms in the Hamiltonian functional and provides a totally explicit discretization of the auxiliary variable without computing extra inner products, which make it more effective and applicable than the traditional scalar auxiliary variable (SAV) approach. To achieve arbitrary high-order accuracy and energy preservation, we utilize the symplectic Runge-Kutta method for both solution variables and the auxiliary variable, where the values of internal stages in nonlinear terms are explicitly derived via an extrapolation from numerical solutions already obtained in the preceding calculation. A prediction-correction strategy is proposed to further improve the accuracy. Fourier pseudo-spectral method is then employed to obtain fully discrete schemes. Compared with the SAV schemes, the solution variables and the auxiliary variable in these ESAV schemes are now decoupled. Moreover, when the linear terms are of constant coefficients, the solution variables can be explicitly solved by using the fast Fourier transform. Numerical experiments are carried out for three Hamiltonian PDEs to demonstrate the efficiency and conservation of the ESAV schemes

Two novel classes of arbitrary high-order structure-preserving algorithms for canonical Hamiltonian systems

In this paper, we systematically construct two classes of structure-preserving schemes with arbitrary order of accuracy for canonical Hamiltonian systems. The one class is the symplectic scheme, which contains two new families of parameterized symplectic schemes that are derived by basing on the generating function method and the symmetric composition method, respectively. Each member in these schemes is symplectic for any fixed parameter. A more general form of generating functions is introduced, which generalizes the three classical generating functions that are widely used to construct symplectic algorithms. The other class is a novel family of energy and quadratic invariants preserving schemes, which is devised by adjusting the parameter in parameterized symplectic schemes to guarantee energy conservation at each time step. The existence of the solutions of these schemes is verified. Numerical experiments demonstrate the theoretical analysis and conservation of the proposed schemes

Improved Noisy Student Training for Automatic Speech Recognition

Recently, a semi-supervised learning method known as "noisy student training" has been shown to improve image classification performance of deep networks significantly. Noisy student training is an iterative self-training method that leverages augmentation to improve network performance. In this work, we adapt and improve noisy student training for automatic speech recognition, employing (adaptive) SpecAugment as the augmentation method. We find effective methods to filter, balance and augment the data generated in between self-training iterations. By doing so, we are able to obtain word error rates (WERs) 4.2%/8.6% on the clean/noisy LibriSpeech test sets by only using the clean 100h subset of LibriSpeech as the supervised set and the rest (860h) as the unlabeled set. Furthermore, we are able to achieve WERs 1.7%/3.4% on the clean/noisy LibriSpeech test sets by using the unlab-60k subset of LibriLight as the unlabeled set for LibriSpeech 960h. We are thus able to improve upon the previous state-of-the-art clean/noisy test WERs achieved on LibriSpeech 100h (4.74%/12.20%) and LibriSpeech (1.9%/4.1%).Comment: 5 pages, 5 figures, 4 tables; v2: minor revisions, reference adde

Multi-Dialect Speech Recognition With A Single Sequence-To-Sequence Model

Sequence-to-sequence models provide a simple and elegant solution for building speech recognition systems by folding separate components of a typical system, namely acoustic (AM), pronunciation (PM) and language (LM) models into a single neural network. In this work, we look at one such sequence-to-sequence model, namely listen, attend and spell (LAS), and explore the possibility of training a single model to serve different English dialects, which simplifies the process of training multi-dialect systems without the need for separate AM, PM and LMs for each dialect. We show that simply pooling the data from all dialects into one LAS model falls behind the performance of a model fine-tuned on each dialect. We then look at incorporating dialect-specific information into the model, both by modifying the training targets by inserting the dialect symbol at the end of the original grapheme sequence and also feeding a 1-hot representation of the dialect information into all layers of the model. Experimental results on seven English dialects show that our proposed system is effective in modeling dialect variations within a single LAS model, outperforming a LAS model trained individually on each of the seven dialects by 3.1 ~ 16.5% relative.Comment: submitted to ICASSP 201